Repository navigation
Conversation
…work test_vm_deployment_planner.py and the VM tests in test_metrics_api.py deploy their VMs as the admin with no network, so they land in the admin account's shared network. When that network's router is stuck, or an earlier test has left the network in Shutdown, every VM in these files fails to start and the smoke run reports failures that have nothing to do with the change under test. Create an L2 network owned by the test account and deploy into it (in advanced zones only for the metrics test, which also runs on basic zones), and wait for usage history stats to show up instead of sleeping a fixed two minutes.
boring-cyborg
Bot
added
component:integration-test
Python
|
sensible test and clgtm, I just wonder if this should be an extra test instead of a replacement. running in an admin shared network makes sense as test as well, does it? |
Thanks @DaanHoogland. I'd lean replacement here. In these two files the network isn't the thing under test, it's just scaffolding to get a VM running so the planner and the metrics APIs can be exercised. Deploying into the caller's default network (the admin shared network on Trillian) couples the test to global state: the admin router being healthy, and no earlier test having left that network in Shutdown, which is exactly what makes them flap (tid-16935, tid-16908, tid-16968). The test-account L2 network removes that coupling and the virtual-router dependency, so these only fail when the planner or metrics code actually breaks. Keeping a shared-network variant of these two would just re-add that flaky coupling to tests that aren't about network placement. If we want explicit coverage that deploying into a shared/admin network works, I think that belongs in its own dedicated test that sets the network up and tears it down, so it isn't at the mercy of leftovers, rather than riding on the planner/metrics smoke tests. Happy to write that as a follow-up if you think it's worth having. Does that sound reasonable? |
Description
test_vm_deployment_planner.pyand the VM tests intest_metrics_api.pydeploy their VMs without an account or a network, so the VMs land in the calling account's default network. On Trillian that is the admin's shared network, and when its router is stuck or an earlier test has left the network in Shutdown, all five planner tests fail with "Unable to orchestrate the start of VM instance" (for example tid-16935 on #13753 and tid-16908 on #13884), and the VM metrics tests fail the same way (tid-16968 on #11968). Both files already create an account, but the deploys never use it.This creates an L2 network owned by the test account and deploys into it (for the metrics test only in advanced zones, since it also runs in basic zones), and in the vms and volumes usage history tests it waits until the history has stats instead of sleeping a fixed two minutes.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
N/A
How Has This Been Tested?
Ran both files with Marvin against a KVM advanced zone. The calls ran as an account that owns more than one isolated network in that zone, which breaks the same dependency on the caller's default network in another way.
test_vm_deployment_planner.pybefore the change, all five tests error in 0.1 seconds:After the change, all five pass (about 134 seconds each) and the VMs are deployed in the test account's L2 network.
test_metrics_api.pypasses on this zone both before and after (14 tests), but before the change the admin VMs were deployed in the admin account's isolatedadmin-networkand the user VM in an isolated network created for the test account on the fly, both behind a virtual router, which is the dependency that fails on Trillian. After the change all four VMs are in the test account's L2 network. Waiting for stats instead of sleeping also shortens the history tests:How did you try to break it?
Checked that the L2 network and its offering are removed when the class is torn down (the class cleanup runs in reverse order, so the network goes before the offering and the account). The zone used here is an advanced zone; in a basic zone the metrics test creates no L2 network and passes no network id, so its deploys are the same as before.